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Demand for More Rigor in Standards 


States have been criticized by reported disparities in outcomes on 
state tests and other educational indicators like NAEP, SAT, ACT, college 
remediation rates, college graduation rates (Lee, 2007, Linn, 2003, 

Linn, 2005) 

Great disparity when comparing the percent of students proficient on 
8 th grade NAEP with state proficiency (...comparing % college ready on 
ACT/SAT with % proficiency in state high school tests) 

External outcomes seem to contradict findings reported from most 
state standards: 

- College Remediation Rates ( over 36 % of undergraduates report taking remedial 
courses) and College Placement Test results (far more students do not meet 
college placement cut scores and may either get a waiver or not enroll). 

- International comparisons show college going students in US losing ground in 
relation to similar students in other nations (OECD, PISA, TIMSS) 

- Anecdotal evidence from the business community, higher ed faculty and 
policymakers supports these findings. 
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Facets of College Readiness 



textual Skill 
& Awareness 



Academic 

Behaviors 


Key Cognitive Strategies -analysis, 
reasoning, interpretation 

Key Content -math, writing, 
reading, science, research 

Academic Behaviors-self 
monitoring, study skills 

Contextual Skills & Awareness- 
knowledge, norms, expectations, of 
college; ability to work and learning 
collaboratively... 
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David T. Conley (2007) 


College and Career Readiness (CCR) 


Preparedness 


Ability to Benefit 


• NAEP Focus on College 
Preparedness, a subset of 
Readiness. 

— Limited to academic 
qualifications; Excludes 
behavioral and contextual 
information 


• Preparedness means qualified 
for entry into postsecondary 
education (college or 
workplace training). 

• NAEP research (occupations 
requiring > 3 mths training 
but less than BA. 


College applicants who have 
not earned a high school 
diploma or GED must pass an 
approved test to qualify for 
federal aid as Ability-to-Benefit 
(ATB) students access to Federal 
Title IV Financial Aid programs. 


Students must demonstrate 
they possess sufficient (ATB) 
from post-secondary education 
via their performance in an 
approved test to justify 
financial aid. 
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Gathering evidence of CCR 


5 


• Task is to make inferences about high school 
students readiness for postsecondary education 
(college, workplace training). 

• Difficulty is that we often are making these 
inferences 2, 3, or 4 yrs in advance. 

- College and Career Readiness Consortiums are asked to 
make even earlier projections (linkages). 

• Predicting future academic behaviors 


• Backmap or sequence postsecondary proficiencies 
(KSAs) to establish a trajectory of skill acquisition. 


- Individual differences, contextual differences. 
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Relevance of the Standards for Educational and 6 
Psychological Testing (1999) to CCR 


• Explicit statement of the purpose - graduate 
high school able to succeed in entry-level, 
credit-bearing academic college courses and in 
workforce training programs. 

• Delineate the aspects of the construct that are 
to be represented - CCSS and assessment 
frameworks (TBD) 

• Develop a scientifically sound validity argument 
to support the intended interpretations of CCR. 
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Relevance of the Standards for Educational and 7 
Psychological Testing (1999) to CCR 


• A validity argument depends on more than one 
proposition. Strong evidence in support of one does 
not diminish the need for evidence to support other 
propositions. 

• A few lines of very solid evidence regarding a 
proposition are better than numerous lines of 
evidence of questionable quality. 

• Interpretation of results should be based on multiple 
sources of convergent and collateral data (and 
understanding of normative, empirical and theoretical 
foundations). 
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Establishing Evidence for CCR 

Validity argument: 

- Content - standards are essential to allow access 
to college material. 

- Construct - standards capture the constructs of 
mathematics or ELA (depth, breadth...) as defined 
for postsecondary education. Conceptual 
framework of constructs is invariance across 
standards and postsecondary work. 

- Criterion-related - students proficient on 
standards are able to access and succeed in entry 
level postsecondary work. 
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Common Core State Standards Evidence - 

Validity Argument 

Content- Alignment to H.Ed. Competencies, 
International Benchmarks; Surveys of Importance, 
Criticality of Skills for entering expectations; Literature 
Reviews 

Construct - Relationship to Other Measures (ACT, SAT, 
Accuplacer, H.Ed. Placement tests); Course Syllabus 
(Brown, 2011 ): 

• Applica (Prereq, Reviewed, Introduced, Subsequent) 

• Importanc (Most, More, Less, Least) 

• Calibrate Standards Statements (Rasch) to common scale 
comparisons across courses can be made in terms of 
degree of alignment. 
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CCR State Assessments - Validity Argument 
for PARCC & Smarter-Balanced 

Content- Aligned to CCSS and H.Ed. Competencies in entry 
college and workplace training courses. 

- Judgmental studies (standard setting). Alignment studies. 

Construct - Relationship to Other Measures (ACT, SAT, 
Accuplacer, H.Ed. Placement tests. International 
assessments). 

- Relationship among benchmarks on CCSA and PISA, TIMSS, SAT, ACT, Probability 
for College Admissions, Placement cut scores (data book); convergent & 
divergent validation evidence. 

Criterion-Related Relationship to College Outcomes 
(Course grades, FGPA, Success in Training Program, Training 
Course grades, Persistence). 


- Classification Accuracy - Placement in Entry level credit bearing coi 


Success in AP/Dual Enrollment courses. 



College Board's Readiness Benchmarks: 

Methodology 

Conducted a with national leaders in K- 

12 and Higher Ed to establish the criteria for college readiness: 

• 65% probability of freshmen college grades of 2.7 (B- or 
higher) 

To set SAT benchmarks - 

• conducted to determine which scores 

correspond to a 65% probability of 2.7 or higher. 

To set PSAT/NMSQ.T benchmarks - 

• conducted to determine which 

PSAT/NMSQ scores (in 11 th and 10 th grades) have a 65% 
probability of obtaining the benchmark scores. 

Will set benchmarks on the new 8 th grade this fall 

using comparable groups in a national norming and scaling study 

• 10 th and 8 th graders each take PSAT/NMSQT and ReadiStep , 

w " ^ Council of thief StaMSthpai Officer? 


Datasets used to Set Benchmarks 
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• SAT Benchmarks and Academic Rigor work (n=68,000) 

- 68,000 students from the entering freshman class of 2007 who 

attended one of 110 colleges participating in the SAT validity study. All 
student had: 

• SAT Scores 

• Self-reported HSGPA 

• Self-reported coursework data 

• College grades and GPA 


• PSAT/NMSQT 11 th Grade Benchmarks (n=585,000) 


- Approximately 585,000 students who completed the PSAT/NMSQT as 
a junior in October 2007 and took their first SAT Exam in March, May, 
or June of 2008. 


• PSAT/NMSQT 10 th Grade Benchmarks (n=710,000) 


- Approximately 710,000 students who completed the PSAT/NMSQT in 
the fall of 2007 as a sophomore and in the fall of 2008 as a junior. 
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Benchmarks for College Readiness: 
Are U.S. Students on Track to Succeed? 



National SAT Cohort College and Career Readiness Rate 

• Among students from the class of 2010 who took the SAT, 43% 
achieved a combined score of 1550 or greater 

• College and career readiness rates differ between students 
with core curriculum preparation and those without 


2010 SAT Cohort - US 50 States + DC 


% College Ready 



College Readiness by Core 
Curriculum Participation 
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20 % 

Students with Core Curriculum Students without Core Curriculum 



*Note: The 1.5MM 2010 SAT tester cohort represent 44% of the 3.3MM total students in the US graduating class of 2010; core curriculum is defined by at least four 
years of English, and at least three years of mathematics, three years of natural science, and three years of social science and history 


College Ready benchmarks and AP examination 

performance 
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% Students Scoring 3 or Above 
on AP Exams 
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■ Met College Readiness Subject Benchmark* 

■ Did Not Meet College Readiness Subject Benchmark* 



English Literature English Language Calculus AB 


Students who met or 
exceeded the SAT subject 
level benchmark score are 
also more likely to 
achieve a score of 3 or 
higher on an AP exam in 
that discipline 


AP Exam Subject 


* In this chart , Critical Reading benchmark was associated with English Literature AP exam score , Writing benchmark with English Language AP exam 
score , and Math benchmark with Calculus AB AP exam score ; based on national 2010 cohort data: N=254 / 873 for AP English Literature ; N=247,821for 
AP English Language; N=177,964for Calculus AB 
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APPENDIX 
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Academic Rigor - In Progress 


• Academic rigor or academic intensity refers to academic 
challenge associated with high school curriculum. 

• Academic intensity is positively related to: 

- the number of credits earned during freshman year 

- FYGPA 

- Lower remediation rates (Adelman, Daniel, Berkovits & 
Owings, 2003) 

• The College Board has developed an academic rigor 
index based on self-reported high school coursework. 


ccssc 



19 


Overview of the Academic Rigor Index 


• The type of courses associated with college success (FYGPA) 
differed by subject area. 

- Math 

• The most advanced course taken (e.g. trigonometry, pre-calculus or 
calculus) in each grade was most associated with college grades. 

- English and social science/history 

• Honors, dual enrollment, and AP Classes were most strongly 
associated with college grades 

- Science 

• Both the number and type of courses mattered (e.g. biology, 
chemistry, and physics) as well as participation in honors, dual 
enrollment, and AP courses were all associated with College grades 

- Foreign/ classical language 

• Both the number of years of study and honors, dual enrollme " 1 


AP participation were most related to college grades 
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Overview of the Academic Rigor Index 

• Academic Intensity is calculated for each of 5 academic 


- English 

- Mathematics 

- Science 

- Social science/history 

- Foreign/classical languages 


• Between 0 and 5 points are awarded within each 
subject area 

• Points were awarded based upon the relationship 
between course participation and FYGPA. 

• The score from each of the 5 subjects (ranging from 0-5) 
is summed to create a 0-25 point scale. 


areas: 
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Distribution of Academic Intensity Scores 
for the Sample (N = 67,644) 
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The Relationship between the Academic Rigor Index (ARI) and FYGPA 
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FYGPA (left Scale) 


Percent Achieving a B- or Better (Right Scale) 


For additional information 
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• The College Board's Common Core State Standards and Assessment Page 
http://professionals.collegeboard.com/policy-advocacy/policy/common-state- 
standards 

• For additional data and research go to: 

http^/prgfessiona[ s. collegeboard. com/data-reports-research 


• Proctor, T. P v Wyatt, J. & Wiley, A. (2010). PSAT/NMSQT® Indicators of College Readiness. 
(College Board Research Report No. 2010-4 http://professionals.collegeboard.com/ 
profdownload/pdf/lOb 2587 PS AT_lndicatorsOfReadiness_WEB_101018.pdf 

• Wiley, A., Wyatt, J. & Camara, W.J. (2010). The Development of a Multidimensional Index 
of College Readiness. College Board Research Report No. 2010-3). http://professionals. 

collegeboard. com/profdownload/pdf/lOb 2084 DevMultiDimenRR WEB 100618.pdf 

• Wyatt, J., Kobrin, J., Wiley, A., Camara, W. & Proestler, N. (in press). Development of a 
College Readiness Benchmark and its Relationship to Secondary and Post- Secondary 
School Performance. 

• Wyatt, J., Wiley, A., Camara, W, J. & Proestler, N. (in press). The development of an index 
of academic rigor for college readiness. 


Related College Board Research Reports 


CCSSC 



